Skip to content

HTTP Notifier implementation#56160

Merged
vincbeck merged 15 commits intoapache:mainfrom
aws-mwaa:ghaeli/http-notifier
Oct 22, 2025
Merged

HTTP Notifier implementation#56160
vincbeck merged 15 commits intoapache:mainfrom
aws-mwaa:ghaeli/http-notifier

Conversation

@seanghaeli
Copy link
Contributor

@seanghaeli seanghaeli commented Sep 27, 2025

Implementation and unit test for HTTP Notifier. Here's the Dag I used to test that it works end-to-end

from datetime import datetime
from airflow import DAG
from airflow.operators.bash import BashOperator
from airflow.providers.http.notifications import HttpNotifier

http_notifier = HttpNotifier(
    http_conn_id="local_server_conn",
    endpoint="",
    method="POST",
    json={
        "dag_id": "{{ dag.dag_id }}",
        "task_id": "{{ task.task_id }}",
        "execution_date": "{{ ds }}",
        "status": "success"
    }
)

dag = DAG(
    "test_http_notifier",
    start_date=datetime(2024, 1, 1),
    schedule=None,
    catchup=False,
    on_success_callback=http_notifier,
)

test_task = BashOperator(
    task_id="test_task",
    bash_command="echo 'Testing HTTP notifier'",
    dag=dag,
)

I add a connection within airflow using:

airflow connections add local_server_conn \ 
--conn-type http \ 
--conn-host host.docker.internal \ 
--conn-port 8081 \ 
--conn-schema http

And spin up a local API server via python:

python3 -c "
import http.server
import socketserver
import json
from datetime import datetime

class Handler(http.server.SimpleHTTPRequestHandler):
    def do_POST(self):
        print(f'[{datetime.now().strftime(\"%H:%M:%S\")}] 🚁 AIRFLOW NOTIFICATION!')
        
        length = int(self.headers.get('Content-Length', 0))
        data = self.rfile.read(length).decode('utf-8')
        
        try:
            json_data = json.loads(data)
            print(f'📋 Airflow sent:')
            print(json.dumps(json_data, indent=2))
        except:
            print(f'📝 Raw data: {data}')
        
        print('=' * 60)
        
        self.send_response(200)
        self.send_header('Content-Type', 'application/json')
        self.end_headers()
        self.wfile.write(b'{\"status\": \"notification received!\"}')

print('AIRFLOW NOTIFICATION SERVER READY')
print('Listening on http://localhost:8081')
print('=' * 60)
socketserver.TCPServer(('', 8081), Handler).serve_forever()
"

Then I run the above Dag, and on the CLI with the python API-server running I can see the contents of the POST request that is triggered when the Dag completes successfully.

@eladkal
Copy link
Contributor

eladkal commented Oct 1, 2025

Notifiers needs to register in provider.yaml
similar to

airflow/providers/amazon/provider.yaml

Lines 908 to 911 in 696fd73

notifications:
- airflow.providers.amazon.aws.notifications.chime.ChimeNotifier
- airflow.providers.amazon.aws.notifications.sns.SnsNotifier
- airflow.providers.amazon.aws.notifications.sqs.SqsNotifier

I guess we are missing verification that any new notifier has corresponded entry in the provider.yaml

@seanghaeli seanghaeli force-pushed the ghaeli/http-notifier branch from d0b2f2b to e652106 Compare October 2, 2025 23:43
@seanghaeli
Copy link
Contributor Author

seanghaeli commented Oct 2, 2025

Bad rebase, please ignore if you were pinged.

jscheffl
jscheffl reviewed Oct 3, 2025
View reviewed changes
Copy link
Contributor

@jscheffl jscheffl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this! No problem being pinged.

As you have in the PR description an example, can you add this example to the codebase as well and a reference into docs so that people can find it? Will be useful for people to know other than taking a look to the source.

Lee-W
Lee-W reviewed Oct 8, 2025
View reviewed changes
@seanghaeli seanghaeli force-pushed the ghaeli/http-notifier branch from d8bf6e2 to 2aa61fd Compare October 14, 2025 18:46
ephraimbuddy
ephraimbuddy reviewed Oct 20, 2025
View reviewed changes
ephraimbuddy
ephraimbuddy reviewed Oct 20, 2025
View reviewed changes
@ferruzzi ferruzzi mentioned this pull request Oct 20, 2025
Closed
1 task
@seanghaeli seanghaeli force-pushed the ghaeli/http-notifier branch from 52411f8 to 09d6f56 Compare October 21, 2025 00:16
@seanghaeli seanghaeli force-pushed the ghaeli/http-notifier branch from 09d6f56 to d6b8606 Compare October 21, 2025 20:23
@vincbeck vincbeck merged commit c3daef9 into apache:main Oct 22, 2025
215 of 216 checks passed
@vincbeck vincbeck deleted the ghaeli/http-notifier branch October 22, 2025 18:28
@potiuk potiuk mentioned this pull request Nov 14, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ephraimbuddy ephraimbuddy ephraimbuddy left review comments

@jscheffl jscheffl jscheffl left review comments

@ferruzzi ferruzzi ferruzzi approved these changes

@Lee-W Lee-W Lee-W approved these changes

@eladkal eladkal Awaiting requested review from eladkal

@o-nikolas o-nikolas Awaiting requested review from o-nikolas

@ashb ashb Awaiting requested review from ashb ashb is a code owner

@kaxil kaxil Awaiting requested review from kaxil kaxil is a code owner

@amoghrajesh amoghrajesh Awaiting requested review from amoghrajesh amoghrajesh is a code owner

@bolkedebruin bolkedebruin Awaiting requested review from bolkedebruin

@potiuk potiuk Awaiting requested review from potiuk potiuk is a code owner

@gopidesupavan gopidesupavan Awaiting requested review from gopidesupavan gopidesupavan is a code owner

@pierrejeambrun pierrejeambrun Awaiting requested review from pierrejeambrun

@bbovenzi bbovenzi Awaiting requested review from bbovenzi

@mobuchowski mobuchowski Awaiting requested review from mobuchowski

@vincbeck vincbeck Awaiting requested review from vincbeck

@jedcunningham jedcunningham Awaiting requested review from jedcunningham jedcunningham is a code owner

@hussein-awala hussein-awala Awaiting requested review from hussein-awala

@uranusjr uranusjr Awaiting requested review from uranusjr

@jason810496 jason810496 Awaiting requested review from jason810496 jason810496 is a code owner

@shahar1 shahar1 Awaiting requested review from shahar1

@romsharon98 romsharon98 Awaiting requested review from romsharon98

@ryanahamilton ryanahamilton Awaiting requested review from ryanahamilton

@shubhamraj-git shubhamraj-git Awaiting requested review from shubhamraj-git

@bugraoz93 bugraoz93 Awaiting requested review from bugraoz93 bugraoz93 is a code owner

@rawwar rawwar Awaiting requested review from rawwar

@XD-DENG XD-DENG Awaiting requested review from XD-DENG

+2 more reviewers

@ramitkataria ramitkataria ramitkataria left review comments

@prdai prdai prdai approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

area:providers provider:http

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

9 participants

@seanghaeli @eladkal @ferruzzi @ephraimbuddy @Lee-W @ramitkataria @prdai @jscheffl @vincbeck